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ACCURACY  OF  INFORMATION  ASSIMILATION  FROM 
UPDATE.)  ALPHA -NUMERIC  DISPLAYS 


In  command,  information  processing  systems  being  considered  for  develop 
meat ,  military  information  will  be  presented  t.j  commanders  and  their  staffs 
to  aid  in  the  making  of  tactical  decisions.  As  a  preliminary  to  the  de¬ 
cision  process,  information  must  be  rapidly  and  accurately  assimilated  from 
displays.  In  particular,  information  which  has  been  updated,  because  of 
changes  in  the  military  situation  may  need  to  be  immediately  apprehended. 

Tue  Command  Systems  Task  of  the  U,  3.  Army  Personnel  Research  Office 
has  initiated  a  series  of  studies  designed  to  provide  information  on  the 
optimization  of  human  performance  in  command  information  processing  systems 
Studies  have  been  concerned  with  t he  p.ssirF lation  of  ir.for  ■ 

mation  from  charts  and  tne  assimilation  of  symbolic  information  from  maps 
and  overlays . 

In  a  recently  completed  study  on  the  assimilation  of  alpha-numeric 
information  (Hammer  and  Ringed,  1964),  subjects  were  required  to  locate  up¬ 
dated  information  by  comparing  updated  charts  with  hard  copy  "history'1. 

The  updates  were  size  coded  in  half  the  charts  presented  and  uncoded  in  the 
remaining  half.  While  amount  of  time  taken  to  locate  updates  was  the  prin¬ 
cipal  dependent  variable,  the  study  yielded  the  following  findings  con¬ 
cerning  errors:  (l)  Two  types  of  error  were  found,  errors  of  omission  and 
errors  of  commission  (in  the  ratio  of  three  to  one).  (2)  Errors  were  re¬ 
duced  by  half  when  coded  updates  were  used.  The  low  frequency  of  errors, 
however,  precluded  a  more  detailed  analysis  of  the  data..  It  was  felt  that 
a  more  stressful  experimental  task  might  yield  more  information  on  how 
accuracy  and  error  scores  are  affected  by  coding,  amount  of  information 
presented,  and  amount  of  information  updated .  Accordingly,  the  present 
experiment  was  designed  to  provide  this  information.  An  additional  purpose 
was  to  determine  if  the  effects  of  the  independent  variables  on  errors  were 
similar  to  those  found  in  a  study  on  the  assimilation  of  symbolic  infor¬ 
mation  (Ringel  and  Vicino,  1964). 

METHOD  OF  INVESTIGATION 


STIMULUS  MATERIAL 

The  conspicuity  technique  used  in  this  experiment — as  in  the  previous 
alpha-numeric  study  (Hammer  and  Ringel,  1964) --was  size  coding,  which 
appeared  to  be  feasible  in  terms  of  system  hardware  capability. 


Charts  used  in  the  experiment  were  adapted  in  f  coast  anc  con tent  from 
charts  contemplated  for  i-.se  in  a-  auT-omatoi.  inf  rmati  u  processing  system. 
They  were  the  same  charts  as  were  used  in  the  previous  study.  Ail  stimulus 
charts  contained  six  columns  of  information:  three  columns  of  words  and 
tnree  columns  of  numbers  with  random  assignee  d  of  iaf'omatijn  within  each 
column.  Each  chart  was  divided  into  blocks  of  equal,  size,  ana  random 
assignment,  of  updates  was  made  to  the  cells  or  cell  within  east  block. 

This  procedure  prevented  Tie  chance  assignment  of  all  updates  to  a  restric¬ 
ted  area  of  a  given  chart,  and.  thereby  reduced  effects  of  position  of  up¬ 
dates  on  performance .  All  uncoded  information  was  ty-.ied  into  the  charts, 
the  coded  information,  prepared  by  "Headliner",  was  pasted  into  the  appropri¬ 
ate  cells.  .Ail  charts  were  then  reproduced  as  35  ram  negative  slides,  figure 
1  is  an  example  of  a  coded  stimulus  chart  with  a  total  of  $0  elements,  of 
which  16  are  updated  coded  elements.  For  each  coded  stimulus  chart  there 
was  an  uncodel  stimulus  chart  identical  is  c-<  ter+.  f  ?v.ny -  <■  har<3 

copy  "history"  answer  chart  corresponding  to  the  s timulus  char-*  in  vigure  1. 
AL1  answer  charts  were  uncoded. 

APPARATUS 

The  experiment  was  conducted  in  a  light -proof ed  room  equipped  with 
rheostatic  ally  controlled  overhead  indirect  light  fixtures.  The  35  mm 
slides  were  rear -projected  to  provide  charts  59"  wide  and  high.  The 
height  and  stroke  width  of  letters  or  numbers  on  the  screen  was  j/ 8"  and 
l/8",  respectively,  for  uncoded  characters  ard  1-3/16'’  vud  l/V',  res¬ 
pectively,  for  coded  characters.  The  average  l-niname  /aiue  of  letters 
and  numbers  was  approximately  1.0  foot  lamberts  'the  sliaes  consisted  of 
transparencies  with  the  alpha-numeric  information  appearing  in  white 
against  a  black  background).  The  subject  was  seated  11"  in  front  of  the 
screen.  Illumination  level  on  the  working  surface  for  the  subject  was 
1.1  foot  candles. 

Tine  subject  was  seated  and  given  a  booklet  containing  instructions 

and  answer  charts.  The  experimenter  read  the  instructions  aloud  and  the 
subject  was  asked  to  read  along  with  him  silently.  Several  practice  trials 
were  then  given  to  acquaint  the  subject  with  the  experimental  task.  These 
trials  were  followed  by  the  experimental  presentations.  The  experimental 
task  was  as  follows :  The  subject  studied  an  uncoded  chart  in  his  booklet 
which  represented  hard  copy  "history".  After  one  minute  he  closed  the 
booklet,  and  was  shown  an  updated  "history"  chart  or.  the  screen.  In  the  up¬ 
dated  chart,  varying  numbers  of  entries  were  different  from  the  corre¬ 
sponding  entries  in  the  hard  copy  "history".  After  one  minute,  the  updated 
chart  was  removed  from  the  screen.  The  subject  then  reopened  his  booklet 
and  was  allowed  one  minute  to  locate  and  cross  out  the  entries  which  had 
been  updated. 
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Figure  1.  Exomple  of  Coded  Updated  Alpha-Numeric  Information 
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Figure  2.  Example  of  Hard-Copy  History  Answer  Chart 


SCORING 


Accuracy  and  error  scores  were  obtained  for  each  trial.  Two  error 
scores  were  obtained,  one  for  errors  of  omission  and  one  for  errors  of 
commission.  An  error  of  omission  was  defined  as  the  failure  to  locate 
an  update.  An  error  of  commission  was  defined  as  the  inaccurate  location 
of  an  update.  Percentages  for  accuracy  and  errors  were  obtained  according 
to  the  following  formulae: 


1.  A  = 


R 


R+O+C 


x  100 


P  - 

o  R+O+C 


x  100 


c 


R+O+C 


x  100 


where : 

A  =  percent  accuracy 
percent  omits 
percent  comnits 
number  correct  responses 
0  =  number  omits 
C  =  number  commits 


VARIABLES 

Independent  Variables 

1.  Coded  vs  uncodea  updated  information 

2.  Total  number  of  elements  of  information  presented.  An  element 
was  defined  as  that  word  or  number  which  appeared  in  a  given  row  and  column 
of  an  alpha-numeric  chart.  The  levels  used  in  the  study  consisted  of  36, 
54,  72,  and  90  elements  presented. 

3.  Number  of  elements  of  information  updated.  The  levels  used 
were  4,  8,  12,  and  16  updates. 
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Dependent  Variable.: 

1.  Accuracy 

2.  Errors  of  omission;  errors  of  commission 


SUBJECTS 


The  subjects  for  this  experiment  were  30  enlisted  men  of  above 
average  intelligence  as  indicated  by  their  scores  for  the  General  Tech- 
nical  Aptitude  Area  (110  or  higher)  of  the  Army  Classification  Battery. 
All  subjects  were  required  to  have  normal  or  corrected  normal  vision. 


EXPERIMENTAL  DESIGN 


The  subjects  were  randomly  assigned  to  two  groups  of  If/  men  each, 
one  performing  with  coded  updates  and  the  other  with  un coded  updates. 

Each  group  took  all  1 6  treatment  combinations  of  elements  presented  and 
elements  updated.  For  purposes  of  administration,  each  group  was  divided 
into  three  subgroups  of  5  each.  Order  cf  administration  of  trials  was 
randomized  for  each  subgroup.  The  design  was  planned  with  the  possibility 
of  combining  the  data  for  the  coded  and.  uncoded  groups  into  one  analysis 
if  the.  variances  proved  to  be  homogeneous.  The  experimental  layout  is 
shown  in  Figure  3. 

RESULTS 

Analyses  of  variance  of  percentages  were  computed  for  accuracy,  omits, 
and  ..oramits,  and  because  of  possible  skewness  of  the  data,  for  percentages 
transformed  to  log  scores.  Results  of  the  analyses  were  essentially  the 
same  and  shoved  all  main  effects  to  be  significant  at  the  .01  or  .05  level 
--with  one  notable  exception^-and  indicated  that  interpretation  could  be 
based  on  the  un transformed  data.  Summaries  of  the  analyses  and  results  of 
significance  tests  on  untransformed  data  are  shown  in  Tables  1,  2,  and  3» 
Figures  4,  5,  6,  and  7  show  accuracy  and  error  performance  across  levels 
of  the  independent  variables .  Summary  tables  of  the  mean  percentages  for 
the  treatment  combinations  are  shown  in  the  appendix  (Tables  A-l  through 
A -6)  • 


While  the  elements  presented  x  elements  updated  interaction  was  signi¬ 
ficant  for  all  the  analyses,  the  "F’s"  were  not  large  and  the  profiles  of 
means  for  this  interaction,  with  the  exception  of  those  for  percent  accuracy 

^  The  effect  on  accuracy  of  number  of  uncoded.  elements  updated  was  not 
significant  with  untransformed  scores  but  was  significant  with  transformed 
scores . 
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with  uncoded  updates,  did  net  reveal  any  meaningful  trends.  A  closer  exami¬ 
nation  of  the  means  for  percent  accuracy  with  uncoded  updates  for  this 
particular  interaction  showed  somewhat  puzzling  results.  Performar.ee 
accuracy  declined  or  remained  stable  for  56,  54,  and  72  elements  presented 
as  the  number  of  unccded  updates  increased  from  4  to  it.  However,  at  the  90 
level,  performance  accuracy  actually  increased  f^om  %  to  16%  (Table  A -2)^. 

A  re-examination  of  the  hard  copy  "history"  answer  charts  and  stimulus 
materials  did  not  reveal  any  scoring  errors  or  artifacts  which  could  have 
produced  this  'unexpected  result. 

Since  the  results  of  an  analysis  of  variance  combining  coded  with  un- 
coded  displays  were  difficult  to  interpret  because  of  heterogeneity  of 
variance,  differences  between  performance  tinier  the  coved  arc:  unccded  con¬ 
ditions  were  tested  for  significance  by  a  test  which  does  r.o*  assume  homo¬ 
geneity  of  variance  ( 0  lx  or.  and  Massey,  '1937  >  ?•  H;  -424)  .  T  a  results  of 
these  tests  are  shown  in  Table  4.  All  dlflcr  ..-..a."*  beyond 

the  .05  level.  With  the  came  test,  differences  between  as  as  jercen ‘ages  of 
omits  and  commits  for  coded  and  uncoded  updates  ware  also  fend  to  be 
significant  beyond  the  .05  level  (Table  5) • 


Table  1 

SUMMARY  OF  ANALYSES  CF  VARIANCE  OF  PERCENT  AC  I'PACY 


Source  of  Variation 

df 

Coded 

MS 

F 

Uncod 

MS 

e& 

F 

Between  Subjects 

14 

2609.30 

904.82 

Within  Subjects 

225 

Elements  Presented  (P) 

3 

7078.99 

11.32** 

loo48. jS 

24.44** 

Subjects  (S)  x  P 

42 

625.44 

411.16 

Elements  Updated  (u) 

3 

11610.15 

15.46**- 

262 . 18 

1.00 

S  x  U 

42 

862.59 

245.20 

P  x  U 

9 

1334.05 

2.55* 

78.3.78 

4.01** 

S  x  P  x  U 

126 

521.62 

196.13 

*  P<  .05 

**  P<  .01 


^  The  effects  of  this  reversal,  combined  with  the  nature  of  the  distribution 
of  percentages,  produced  the  significant  "F"  for  the  main  effect  for  trans¬ 
formed  scores  and  showed  performance  accuracy  actually  to  increase  with 
increasing  numbers  of  uncoded  updates. 
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Table  2 


SUMMARY  OF  A1IALYSES  OF  VARIANCE 


OF  FERCELUAGES  OF  OMITS 


Source  of  Variation 

df 

Coded 

MS 

F 

Uncoded 

MS  F 

Between  Subjects 

14 

1116.61 

3414.38 

Within  Subjects 

225 

Elements  Presented  (P) 

3 

4 137.40 

13.33** 

4822.13 

7.55** 

Subjects  (s)  x  P 

42 

310.87 

638.98 

Elements  Updated  (u) 

3 

8894.47 

30.85** 

6751.98 

14.32** 

S  x  U 

42 

295.6* 

471.41 

P  x  U 

9 

834.01 

3.51** 

71303 

2.04* 

S  x  P  x  U 

12  6 

23  ? . 49 

349.51 

*  P<  .05 
**  P<  .01 


Table  3 

SUMMARY  OF  ANALYSES  OF  VARIANCE  OF  PERCENTAGES  OF  COMMITS 


Source  of  Variation 

df 

Coded 

MS 

F 

Uncoded 

MS  F 

Between  Subjects 

14 

517-11 

2270.58 

Within  Subjects 

22£ 

Elements  Presented  (P) 

3 

760.66 

5.60* 

936.57 

5.98** 

Subjects  (S)  x  P 

42 

221.55 

156.73 

Elements  Updated  (U) 

3 

1509.68 

6.03** 

4103-42 

19.83** 

S  x  U 

42 

250.44 

206.86 

P  x  U 

9 

328.33 

1.97* 

356.08 

2.12* 

S  x  P  x  U 

126 

166.93 

168.18 

*  P<  .05 
**  P<  .01 
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Table 
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DIFFERENCES  BETWEEN  PERCENTAGES  OF  SIDED  AND  UNCODED  UPDATES 


Coded 

Uneoded 

Significance 

Level 

Accuracy 

73 

24 

.01 

Omits 

16 

58 

.01 

Commits 

11 

i8 

•05 

Table  5 

DIFFERENCES  BETWEEN 

PERCENTAGES  OF  OMITS  AND  COMMITS 

Omits 

Significance 

Commits  Level 

Coded 

16 

11  .05 

Uncoded 

58 

18  .01 
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Figure  6.  Percent  Accuracy  for  Coded  and  Uncoded  Updates  at  Each 
Level  of  Elements  Updated. 
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Percent  Errors  of  Omission  and  Commission  for  Coded  and 
Uncoded  Updates  at  Each  Level  of  Elements  Updated. 


The  result-;  of  this  -  ;udy  may  be  summarized  as  follows: 

A;  the  nuiut- .  j  ■  Lei  t  of  '  rmation  Increased  i  roc  to  JO: 

1.  A.  :urac  .  fi am  8 6$  •- ;  62$  for  <  5ded  updates  and  from 

38$  to  11$  for  encode ’  at  .  tes  (Figure  4). 

2.  Errors  of  omission  increased  from  7$  to  24$  fer  coded  updates 
and  from  49$  to  67$  for  unccded  up'date:--  (Figure  5)* 

3.  Error;  of  -ommission  increased  from  7$  to  l4$  for  coded  updates 
and  from  13$  to  1  $  1  or  ided  g  iates  (Figure  3) . 

As  the  number  of  elements  of  informal  ..or.  updated  increased  from  4  to  lo 


“T.  Accur 


- _ JL  _  O -  .  -  -  _  .  .-.1)  _ 


dates  but 


mined  approximately  ttabl  at  22$  for  uncoded  updates  (Figure  6).  (This 
apparent  stability  of  accuracy  scores  for  .rc  -.-ed  opiates  may  be  somewhat 
suspect  in  view  of  the  significant,  element-;  prcseiited  x  elements  updated 
interaction) . 

2.  Errors  of  omission  increased  from  6$  to  28$  for  coded  updates 
and  decreased  from  27$  to  11$  for  unccded  updates  (figure  7) . 

3.  Errors  of  commission  increased  from  6$  to  l7$  for  coded  updates 
and  decreased  from  27$  to  11$  for  uncoded  updates  (Figure  7). 

The  following  major  differences  were  food  as  a  function  of  the  use  of 
coded  vs.  uncoded  updates  respectively  (Table  4): 


1.  Performance  accuracy  -  73$  vs.  24$. 

2.  Errors  of  omission  -  io$  vs.  38$. 

3.  Errors  of  commission  -  11$  vs.  18$. 

The  percentages  of  omits  were  significantly  larger  than  the  percentages 
of  commits  for  both  coded  and  uncoded  updates  (Table  5) • 

Comparison  of  the  results  of  the  present  study  with  those  of  a  previous 
study  on  the  assimilation  of  symbolic  information  {.Ringel  and  Vic i no ,  lo64) 
indicate  the  following : 

1.  Elements  (or  amount)  of  information  presented  affected  perform¬ 
ance  accuracy  similarly  in  both  studies. 

2.  In  both  studies,  errors  of  omission  occurred  with  greater  fre¬ 
quency  than  errors  of  commission. 


iMPL:y> 


The  enhancement  of  ec-u’forjiar.i.oe  a  *eur' 


;  V 


ceding  up¬ 


dated  information  supports  previous  findings  for  the  v.inlj.®lon  of  a  coding 
capability  in  proposed  information  processing  :rr  'gel.  •’ Hammer  and  Ringed, 
196*0.  For  information  processing  tasko  .  inilar  to  chat  rc-i  in  the 
current  study  the  partie.iiar  coding  cechni'pue  used  (size  coding)  is  feasible 
from  a  standpoint  of  systems  he.rdvs.re  eapab ility ,  and  may  be  as  effective 
as  other  techniques  which  are  more  difficult  and  error.:  re  to  install. 


The  effects  on  performance  of  the  number  of  elements  presented,  and 
number  of  elements  updated  suggest  that  unless  new  te  .-nniques  for  present¬ 
ing  and  assimilating  alpha-numeric  information  are  developed,  _ixits  may 
need  to  be  set  on  amount,  of  information  presented  and  vpinv.-d  on  any  one 
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in  the  case  of  uncoded  updates ,  the  profiles  of  means  for  the  ele  - 
men to  presented  x  elements  updated  interaction  showed  that  when  90  ele¬ 
ments  were  presented,  performance  accuracy  act* jail;/  in  ■  ceased  as  the  number 
of  elements  updated  was  increased  from  4  to  16.  Thi  .acex-oealed  finding 
might  imply  that  at  this  level  of  difficulty,  subject  .  used  a  different 
information  processing  technique.  However,  before  %  more  definitive 
interpretation  can  be  attached  to  these  data,  reel  .-avic-jL  and  further 
study  are  needed. 


The  data  for  9  of  the  15  subjects  who  worked  with  encoded  displays 
indicate  that  as  the  number  of  uncoded  updates  increased  from  4  to  16,  the 
combined  proportions  for  accuracy  and  commits,  which  represent  total  overt 
responses,  decreased  while  proportions  of  omits  increased.  Coupled  with  the 
apparent  stability  in  performance  accuracy  (assuming  that  the  significant 
elements  presented  x  elements  updated  interaction  did  not  mask  a.  significant, 
main  effect),  these  findings  may  reflect  an  unrealistic  increase  in  the  per¬ 
ceived  difficulty  of  the  experimental  task — an  increase  which  may,  in  turn, 
relate  to  a  reluctance  on  the  part  of  subjects  tc  risk  making  errors .  A 
finding  which  may  support  this  conjecture  emerged  from  a  study  on  assimila¬ 
tion  of  symbolic  information  (Andrews  and  Ringei,  1964)  in  which  confidence 
was  actually  measured  rather  than  inferred.  The  findings  of  both  studies 
point  out  a  need  for  research  on  the  relationships  among  perceived  diffi¬ 
culty,  confidence,  and  performance ,  and  the  implication  of  these  relation¬ 
ships  for  decision  making. 
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APP2K  A 

Tables  Shoving  Means  of  Accuracy  arl  Err.  S:  res 
for  a.’^.  Treatment  Goatl'-A'l  *-as 

Table 

A-l.  Percent  Accuracy  Coded  displays 

A-2.  Percent  Accuracy  Uncodex  lisplays 

A -.5.  Percent  Err  re  of  Omission --Coded  Display' 

A -4.  Percent  Err  irs  •  f  OjpJLj  .  1  ••  .  to  xied  x»i  a  > 

A~‘j.  Percent  Errors  Ccaaaiissicsa— Ceded.  plAys 
A -6.  Percent  Errors  erf  Cornu!  si  n  v to  led  '  j  Ay 
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24 
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Table  A-l 


PERCENT  ACCURACY— CODED  DISPLAYS 


4 

Elements 

8 

Updated 

12 

16 

Mean 

36 

97-30 

95-97 

79.48 

73.87 

86.90 

Elements 

54 

87.33 

79-88 

78.07 

59-64 

76.73 

Presented 

72 

87.67 

90.30 

48.09 

55.25 

71.77 

90 

82.38 

74.77 

56.84 

36.51 

62.87 

Mean 

88.42 

85.48 

65.12 

56.56 

73.39 

Table  A-2 

PERCENT  ACCURACY  —UNCODED  DISPLAYS 

Elements  Updated 

4 

8  12 

16 

Mean 

36 

46.01 

33.85  41.22 

31.96 

38.64 

Elements 

54 

26.20 

25.42  27.87 

25.65 

26.79 

Presented 

72 

29.21 

22.25  13.72 

18.29 

20.63 

90 

3.96 

11.41  14.07 

16.54 

11.01 

Mean 

26.08 

23.97  24.71 

23.35 

2^7rf 
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a&o-i.-  A  -3 


PERCENT  ERRC 

?£  OF  OMISSION - -CODED 

D^SP  j_AYS 

4 

Elements  Updated 

8  ’  12 

16 

Mean 

56 

1.35 

2.48  11.44 

15.14 

7.85 

Elements 

54 

6.78 

10.97  15.27 

26.64 

14.17 

Presented 

72 

5*33 

5.48  54.45 

30.16 

18.36 

90 

10.01 

15.63  29.80 

44.30 

24.94 

Mean 

6.t>l 

7.15  22.24 

28.31 

16.83 

Table 

FERCHNT  ERRORS  OF  OMISI 

A -4 

SION  -  •  UNCOUED 

DISPLAYS 

4 

Elements 

8 

Updated 

12 

3.6 

Mean 

36 

34.55 

52.66 

53.19 

6O.6C 

5C.4S 

Elements 

54 

48.01 

51.31 

57-73 

63.O6 

55.86 

Presented 

72 

36.61 

63.89 

70.49 

74.77 

61.93 

90 

68.56 

64.39 

65.96 

72.54 

67.10 

Mean 


47.68 


57-44 


61.10 


66.99 


Table  A -5 


PERCENT  ERRORS  OF  COMMISSION  -  -CODE'S-  DISPLAYS 


4 

Elements  Upoa 
8 

ted 

12 

16 

Mean 

36 

1.30 

4.55 

10.07 

14.99 

7.23 

Elements 

52* 

7.89 

11.20 

7.64 

16 . 7  > 

10.12 

Presented 

72 

8.00 

5.22 

18.44 

15.58 

12.56 

90 

8.59 

14.37 

15-33 

■-L  .i-y 

Mean 

6.95 

8.39 

15.62 

i6 . 12 

ir:n 

Table  A -6 

PERCENT  ERRORS 

OF  COMMISSION 

—UNCODED 

DISPLAYS 

Elements  Updated 

4 

8 

12 

16 

Mean 

36 

21.45 

16.49 

5.30 

10.45 

13.67 

Elements 

54 

26.79 

24.77 

16.40 

12.27 

20.56 

Presented 

72 

33.18 

15.06 

18.77 

8.9O 

19.48 

90 

50.48 

24.20 

21.98 

13.90 

22.89 

Mean 

27.23 

20.63 

15.II 

11.63 

17TI5 
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